FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

Precursor ion scanning for detection and structural characterization of heterogeneous glycopeptide mixtures

AbstractThe structure of N-linked glycans is determined by a complex, anabolic, intracellular pathway but the exact role of individual glycans is not always clear. Characterization of carbohydrates attached to glycoproteins is essential to aid understanding of this complex area of biology. Specific mass spectral detection of glycopeptides from protein digests may be achieved by on-line HPLC-MS, with selected ion monitoring (SIM) for diagnostic product ions generated by cone voltage fragmentation, or by precursor ion scanning for terminal saccharide product ions, which can yield the same information more rapidly. When glycosylation is heterogeneous, however, these approaches can result in spectra that are complex and poorly resolved. We have developed methodology, based around precursor ion scanning for ions of high m/z, that allows site specific detection and structural characterization of glycans at high sensitivity and resolution. These methods have been developed using the standard glycoprotein, fetuin, and subsequently applied to the analysis of the N-linked glycans attached to the scrapie-associated prion protein, PrPSc. These glycans are highly heterogeneous and over 30 structures have been identified and characterized site specifically. Product ion spectra have been obtained on many glycopeptides confirming structure assignments. The glycans are highly fucosylated and carry Lewis X or sialyl Lewis X epitopes and the structures are in-line with previous results. [Abbreviations: Hex–Hexose, C6H12O6 carbohydrates, including mannnose and galactose; HexNAc—N-acetylhexosamine, C8H15NO6 carbohydrates, including N-acetylglucosamine and N-acetylgalactosamine; GlcNAc—N-acetylglucosamine; GalNAc—N-acetylgalactosamine; Fuc–Fucose; NeuAC—N-acetylneuraminic acid or sialic acid; TSE—Transmissible Spongiform Encephalopathy.

Some interactions among driver, vehicle, and roadway variables in normal driving

Effects of road and vehicle conditions, visual warning signs, direction of turns, night time, and skill on automobile driver performance are studied in several experiments. Considered criteria are variability in speed and acceleration

Movie of the interplanetary magnetic field

Description of movie representing IMP-1 MAGNETOMETER observations of interplanetary magnetic fiel

Interplanetary magnetic field IMP-1, motion picture of the transverse components

Motion picture report of IMP-1 magnetometer observations of interplanetary magnetic fiel

Free induction decay of a superposition stored in a quantum dot

We study the free evolution of a superposition initialized with high fidelity in the neutral-exciton state of a quantum dot. Readout of the state at later times is achieved by polarized photon detection, averaged over a large number of cycles. By controlling the fine-structure splitting (FSS) of the dot with a dc electric field, we show a reduction in the degree of polarization of the signal when the splitting is minimized. In analogy with the "free induction decay" observed in nuclear magnetic resonance, we attribute this to hyperfine interactions with nuclei in the semiconductor. We numerically model this effect and find good agreement with experimental studies. Our findings have implications for storage of superpositions in solid-state systems and for entangled photon pair emission protocols that require a small value of the FSS